LLM Manager

The LLM Manager allows admins to create and govern LLM Providers within Pyramid. The providers are used to generate images, scripts, excel formulas, translations, and to choose colors using AI. Providers can be added, removed, and edited from this page.

You should create or edit the LLM details in this page to reflect those details associated with the actual service or deployment that is already set up in advance. For more information about selecting the LLM that is most appropriate to your needs, and setting it up ready for use, see Integrating LLMs with Pyramid.

LLM Manager

Administrators can access the LLM Manager settings in the Admin Console from AI > LLM Manager:

This page contains the details of all current LLM and speech providers at the top and an Edit LLM panel at the bottom:

  • To view and edit details of an existing LLM in the bottom panel, either select it in the LLM Providers list or click Edit from its Actions (purple arrow below).
  • To create a new LLM Provider in the bottom panel, click Add LLM Provider (yellow arrow). For details, see Adding and Editing LLM providers (below).
  • To create a new Speech Provider in the bottom panel, click Add Speech Provider (blue arrow). For details, see Adding and Editing Speech providers (below).

LLM Providers list (Top Panel)

The LLM Provider Manager lists the details for the providers:

  • Name: The name of the instance; typically matches the provider name.
  • Type: The name of the vendor.
  • Endpoint: The endpoint defined for the LLM provider.
  • Tenant: If the Secure by tenant option is used to restrict this endpoint to a specific tenant, the name of the tenant is shown.

For each of the current providers administrators can click:

  • Edit: To open and edit the given provider details in the bottom panel.
  • Delete: To delete the selected provider.
  • Copy: To create a copy of the selected provider. You will be prompted to provide a new name for your newly created (copied) provider.
  • Audit Trail: To display an audit trail for the selected provider in a pop-up window including operation, performed by, date, and details.

Edit LLM (Bottom Panel)

Common settings

  • Vendor: Select the required vendor from the dropdown list.
  • Name: Provide a name for the LLM provider.
  • Secure by tenant: Select this checkbox if the endpoint should only be available to a specific tenant. Then select the tenant from the dropdown list to associate it with this LLM.

LLM Providers

To create a new LLM Provider in the bottom panel, click Add LLM Provider from the overhead menu and supply the following attributes:

OpenAI

If OpenAI is selected, then enter the following details:

  • API Key: Enter your API key, which can be found in the Account Settings of your OpenAI account.
Azure AI

If Azure AI is selected, then enter the following details:

  • API Key: Enter your API key, which can be found in your Azure portal in the Endpoint and Keys pane of the Resource Management section.
  • Endpoint: Enter your Azure Endpoint URL, which can also be found in your Azure portal.
  • Deployment: Enter the unique name of the deployed model.
  • Model Provider: Select the name of the organization that created or published your AI model. Where the Model Provider is Open AI, capture the following additional values:
    • Routing (Sometimes called the Type.)
    • API Version
    • Base Model
Google Gemini

If Google Gemini is selected, then enter the following details:

  • Region: Select the region providing the service from the dropdown list.
  • Project ID: Enter the project ID for the provider.
  • Model ID: Enter the model ID for the provider.
  • Client ID: Enter the client ID for the provider.
  • Client Secret: Enter the client secret for the provider.
  • Click Sign-In with Google. The OAuth refresh code field is populated in response to this action.
Mistral

If Mistral is selected, then enter the following details:

  • API Key: Enter your API key.
  • Model ID: Enter the model ID for the provider.
IBM Watsonx

If IBM Watsonx is selected, then enter the following details:

  • Region: Select the region providing the IBM Watsonx service from the dropdown list.
  • Project ID: Enter the project ID for the provider.
  • Model ID: Enter the model ID for the provider.

WatsonX needs to be licensed separately. For more information, click here or contact sales.

Speech Providers

To create a new Speech Provider in the bottom panel, click Add Speech Provider from the overhead menu and supply the following attributes:

OpenAI-Whisper

If OpenAI-Whisper is selected, then enter the following details:

  • OpenAI API Key: Enter the required API key.
  • Voice: Select the desired voice type to be used.
  • Voice Speed: Adjust the voice speed.
Google

If Google is selected, then enter the following details:

  • Upload Service Account JSON file: Upload your JSON Service Account file.
  • Recognizer Name: Select the required name for the Google service from the dropdown options that will be populated after selecting a valid JSON file.
  • Voice Gender: Select the gender of the voice to be used.
  • Voice Selection: Select the voice to be used.
  • Voice Speed: Adjust the voice speed.
  • Voice Pitch: Adjust the voice pitch.
  • Voice Volume: Adjust the voice volume.

Note: The Chatbot's speech to text feature is only available with Enterprise licensing.

Macro Functions

Test

Click Test to test the provider details in the Edit LLM panel. In the case of the Speech Providers, the test will help you to ensure you are satisfied with your selected voice, speed, and (in the case of Google) other settings.

Note: The LLM Provider test will fail if the main Generative AI and Natural Language Querying option is not enabled at the top of the AI Settings panel.

Save

Click Save or Apply to save your changes. Save is offered when you are creating a new LLM provider. Apply is offered when you are editing details of an existing LLM Provider.